Indirect reciprocity undermines indirect reciprocity destabilizing large-scale cooperation

Significance The emergence of large-scale cooperation remains one of the great scientific puzzles across many disciplines. Previous models have suggested that indirect reciprocity is sufficient to sustain large-scale cooperation, but these models assume that people only belong to one cooperative group. In reality, people belong to multiple cooperative groups with different, often competing incentives. Here, we extend these models of indirect reciprocity showing that under a range of realistic conditions, reputation at a lower scale of cooperation (smaller group) will undermine reputation at a higher scale of cooperation (larger group).


Supporting Information Text
In the supplementary information text we go over how we derive all the functions which allow us to analyze our model.The actual analysis is completed in the python script also provided in the supplementary information.A full table of results can be found in the spreadsheet, also found in the supplementary information.
We begin by writing the payoffs for each strategy.These strategies can be found in tables S1 and S2.There are a total of 6 × 6 = 36 strategies.Note strategies LG, M , O and p are omitted from the main text and only considered here.For our purposes we need to define a number of functions, which are listed in table S3.Theses functions are dependent on the parameters of our model, listed in table S4.To begin recall the payoff formulas as stated in the methods of the manuscript: F (i, j) = Fp(i, j) + Fm (i, j) [1] Fm(i, j) = (1 − e) (n l − 1) n l (I(i, j)bm − H(i, j)cm) [3] Equations 2 and 3 both rely on defining more functions, specifically related to questions of reputation.Equation 2 relies on defining whether or not someone contributes to a public good (V , Vg, and V l ).Note equation 2 can be written in a more general form where all strategies are checked to see if they give contributions (instead of say only taking Vg for strategies using M in the PGG), but Vg and V l will be either 1 or 0 for all strategies except M , O and LG, so other strategies can be excluded.
Equation 3 relies on defining who a player receives aid from (I(i, j)) and who players provide aid to (H(i, j)), as determined by reputation.
Note when we run the analysis we assume n l −1 n l ≈ 1 in equation 3.This makes the analysis easier to interpret without changing the main insights.It should be noted that for sufficiently small local groups (small n l ) this assumption might be unfair.
We suspect that if groups are small enough then the returns from the MAG are lessened, potentially dissuading cooperation, but for the purposes of this model we ignore this possibility.
We denote three different types of PGG standing, one in the global PGG (denoted by Vg (i, j)), another in the local PGG (denoted by V l (i, j)) and finally an overall PGG standing (denoted by V (i, j)), where the overall PGG standing is the sum of these other two numbers (V (i, j) = Vg (i, j) + V l (i, j)).PGG standing is defined for a given round n, denoted by a superscript, and is reliant upon MAG actions, specifically W (i, j) which will be defined shortly.Note that the important variable of the function is the PGG strategy and we will explicitly define the result for each strategy.
r in the V (LG, j) functions is the ratio that LG players give to each PGG pool.For instance, if r = 0.25 then players of strategy LG give a quarter of their endowments to the global PGG and three quarters to the local PGG.
And now to define W n (i, j).We define this recursively where everyone begins in good standing, or W 1 (i, j) = 1.
And this can be solved at equilibrium by setting W n (i, j) = W n−1 (i, j): Here lost n (i, j) and gained n (i, j) represent the proportion of players using strategy (i, j) who had a good reputation and lost it in round n or had a bad reputation and gained a good one in round n, respectively.Without the superscript n, gained(i, j) and lost(i, j) represent the equilibrium gain and loss of reputation.Note that this is dependent on which of the leading eight strategies of indirect reciprocity which we choose to employ.Each of the leading eight strategies can be calculated the proportion of good players someone aids and defects as well as the proportion of bad players someone aids and defects.
Here X(i, j) is the proportion of players which contributed to the MAG and that players with strategy (i, j) will try to give to, Y (i, j) is the proportion of players which contributed to the MAG that players with strategy (i, j) will not try to give to, and Z(i, j) equals all players that players with strategy (i, j) will try to give to, regardless of MAG reputation.These can then be defined for each MAG strategy as follows.
For MAG strategy c: For MAG strategy g: For MAG strategy l: For MAG strategy d:

Eric Schnell and Michael Muthukrishna
Note that this means W (i, d) = 0.
For MAG strategy p: For MAG strategy m: For MAG strategy pm: These functions are circular and dependent on themselves, with W (i, j) being dependent on X(i, j), Y (i, j) and Z(i, j) and these being dependent on W (i, j).To break this cycle, we assume all W (i, j) in one of these functions is 1, except for MAG defectors in which case it is 0.
Next we can define our functions gained(i, j) and lost(i, j) by different leading eight strategies.
For strategy 1: For strategy 2: For strategy 3: For strategy 4: For strategy 5:

of 11
Eric Schnell and Michael Muthukrishna For strategy 6: For strategy 7: For strategy 8: In returning back to our function of MAG fitness (3), the only remaining undefined functions are G (i, j) and H (i, j).Recall H (i, j) is defined as the proportion of players who someone of strategy (i, j) aids.This can now be defined easily by using the other functions we have defined above as being: Defining how much aid a player of strategy (i, j) receives, or I (i, j), is somewhat trickier.Here we define it as the sum of aid received from each strategy found in our game: This provides us with all the necessary information to conduct our invasion analysis.The invasion analysis is conducted by comparing resident fitness to invader fitness.These fitnesses are found in equation 1.The PGG component of fitness (2) requires defining PGG behaviour which is itself defined in 4. The MAG component of fitness (3) requires defining MAG reputation, which is built up using the remaining listed functions.

Model Discussion
Because of our invasion analysis, there are only ever two strategies present.This simplifies many of the listed equations, but we chose to write these in their general form above.This also explains why strategies using LG fair poorly in our model.Consider a group where there are only two strategies (G, g) and (LG, g).Looking at equation 9 we see that the amount of aid received by each playstyle can be simplified as I(G, g) = yG,gVg (G, g) + yLG,gVg (G, g) and I(LG, g) = yG,gVg (LG, g) + yLG,gVg (LG, g).
In comparing these two strategies we notice that the only difference is in Vg(G, g) versus Vg(LG, g), but we know Vg(G, g) = 1 and Vg(LG, g) = r < 1.So without any more details, we know players using (G, g) will always receive more aid than those using (LG, g).In comparing all strategies a similar pattern emerges where players using LG don't receive as much aid as those using G or L. In reality there are likely to be more than 2 strategies being used at any given time, which allows LG to hedge its bets and potentially out compete those who commit fully to one of the scales.But because of the nature of our analysis that nuance is lost and thus LG is unsustainable.
As for strategies M and O, as we see in the formulas above, these are dependent on having received MAG aid.We find that these strategies are more resistant to defectors because if they stop receiving aid from defectors they will in turn stop subsidizing these defectors.However, when invading or being invaded by pure cooperators they perform worse.The reason for this is the implementation error in providing MAG aid.If we take as an example a player using M , who gives to the global PGG when they've been aided.Even if all players want to aid global cooperators, they will sometimes fail to do so because of the implementation error.In such cases, if the M player doesn't receive aid, then the next round they will purposely defect from the PGG and then again fail to receive MAG aid.Because the implementation error also makes it that sometimes a player gives aid when they didn't mean to, there's a chance that the M who didn't receive aid once and started defecting could later on receive aid and restart cooperating.Otherwise, we may expect all players using a strategy of M and O to eventually defect because of one error.This means that M and O will provide less to the PGG than G and L and will receive less aid in the MAG, and thus under most conditions we find that the more cooperative strategies beat their stricter counterparts.A more lenient form of M and O which only defects if they weren't aid two rounds in a row may avoid this defection trap, but this falls outside the scope of our current model.

Eric Schnell and Michael Muthukrishna
Finally, the model results were not greatly changed by considering different leading eight strategies.As can be seen in the equations above, the strategy we used (strategy 1) is more lenient than certain others seeing as there's only one way to lose standing.Strategies such as 8 are especially harsh seeing as there's only one way to gain standing and multiple ways to lose it.In our analysis, we find that the results remain the same for the majority of strategies and only change for intermediate strategies which are themselves out competed.
In particular, if we focus on the standing (L1), consistent standing (L2), and staying (L7) strategies, which are highlighted by Hilbe et al. (2018) as being the most resilient to noisy reputations, then when assigning specific parameters the only difference in the single invader analysis occurs for strategies (D, pm) and (LG, pm) and in the group level analysis for strategy (LG, pm).
Given that these strategies are themselves outcompeted by others, for instance in the single invader model (D, d) outcompetes both (D, pm) and (LG, pm), whether these strategies themselves invade or are invaded by others doesn't change the main results of the model.This is true for all strategies which are changed by using different leading eight strategies.Seeing as the stable evolutionary endpoints remain the same accross strategies, we can say that changing which leading eight strategy will not change the results of the model, only how those results are reached The analysis was conducted in Python using the SymPy symbolic mathematics computer algebra system (CAS) package and the code for running the analysis can be found in the supplementary information.The full results of the analysis are compiled in an excel spreadsheet, also found in the supplementary information.
Note some of the results are re-evaluated using Mathematica because SymPy was unable to tell whether it was greater or less than 0. For example, (G, l) is able to invade (G, c) in a single invader model when cm • (1 − e) > 0. Based on our assumptions about our parameters (cm > 0 and 0 < e < 1) we know this invasion will always occur, but SymPy was unable to detect it as such so we make this change in the results.

> cp bg
Returns from global PGG contributions

> cp bm
Returns from being aided in the MAG

> cm e
MAG implementation error rate 0 < e < 1 r Ratio of aid provided to the global PGG as a player using PGG strategy LG 0 < r < 1

Table S1 . PGG Strategies PGG strategy Description
M Contribute to Global PGG only if you've been aided in the MAG O Contribute to Local PGG only if you've been aided in the MAG Eric Schnell and Michael Muthukrishna

Table S2 . MAG Strategies MAG strategy Description c
Always aid g Aid if partner contributed to Global PGG l Aid if partner contributed to Local PGG d Always defect p Aid those in good reputation based on PGG contributions m Aid those in good reputation based on MAG contributions pm Aid those in good reputation based on PGG contributions and MAG contributions 8